1. Field of the Invention
This invention relates generally to optical image processing and more particularly characterizing images optically in a way that can be encoded, stored, searched, retrieved, and/or compared to other images.
2. State of the Prior Art
There are hundreds of millions, perhaps billions, of creative works in the world which are in the form of visual images or which can be converted into visual images. Such creative works may include, for example, artistic drawings and paintings, commercial art, technical drawings, photographs, motion pictures, digitally recorded still and video motion images, radar images, maps, computer-generated images, literary works, graphics software code, sound recordings, and many more. Such creative works are stored in libraries, data bases, public and private collections, and other places all over the world, and there are many reasons for persons wanting to find them. However, images are much more difficult than, for example, text, to characterize in an objective manner, independent of human viewing and subjective evaluation. Therefore, creation of a comprehensive data base for storing and searching large numbers of images has been a daunting task and the results have limited utility, even when augmented by computerized indexing and searching. For example, a visual search engine implemented by ditto.com (formerly Arriba Soft Corp.) uses a xe2x80x9ccrawlerxe2x80x9d computer software program to travel the World Wide Web, visiting web sites in search of images, and capturing the images along with any related text. The images are then reduced in size, indexed with the text according to some category of subject matter, content, characteristic, or the like, screened and/or chosen by humans (xe2x80x9chuman filtersxe2x80x9d); and entered into a data base. Subsequent access or searching of the data base is done by entering a key word or phrase, which has to be one of a number of key words or phrases that the search engine architect or operator has chosen to characterize an image in the search engine index. For example, a searcher could enter the word xe2x80x9cbutterflyxe2x80x9d and, if the data base search engine has images indexed by the word xe2x80x9cbutterfly,xe2x80x9d the search engine will display those images. If the search under xe2x80x9cbutterflyxe2x80x9d yields too many hits, e.g. over 3000 images, another word, such as xe2x80x9cmonarchxe2x80x9d can be added to narrow the search field, but only if the data base architect or operator has images indexed under the word xe2x80x9cmonarchxe2x80x9d. The words xe2x80x9cmonarch butterflyxe2x80x9d may narrow the field t to, e.g., several hundred images. It is then up to the human viewer to look at all the images catalogued under the words xe2x80x9cmonarchxe2x80x9d and xe2x80x9cbutterflyxe2x80x9d to see if there is one or more of interest. Ultimately, the search capability of such a search engine is limited by: (i) the word(s) the data base architect or operator choose to describe an image; (ii) the limited time that real people have to view the images, assign key words for indexing, and enter the image with the chosen index words in the data basexe2x80x94currently up to several million images; (iii) the limited images from the hundreds of millions or billions available that the operator decides to index and put into the data base; (iv) the user has to personally view the images presented; and (v) if the user wants to find the source or location of an image of which the user has a reference image that the user has in his/her possession, the user has to hope the search engine operator did not exclude the wanted image, and the user has to compare the images delivered by the search engine to the reference image.
Applicant""s co-pending patent application, U.S. Ser. No. 09/326,362, filed Jun. 4, 1999, now U.S. Pat. No. 6,445,822 describes how images can be found in various data bases, servers, web sites, and the like that are accessible by or via the Internet and compared to a reference image using optical correlation techniques. However, it is not very desirable or efficient to have to search through all such available sources each time a match is sought for a different reference image. Of course, all of the images found in all such available sources could be placed into a single or central data base, and then each search for a match to a reference image could be searched in that data base. Such a scheme would eliminate having to go to all available sources for each search. However, many images require hundreds or thousands of kilobytes of data, so it would require a massive data storage memory to gather and store the hundreds of millions or billions of images together in one data base, and searching through all such images for every reference image would still be inefficient, even if all the images were in a single data base. Yet, attempts to limit searching time and resource requirements by arbitrary categories, while possibly beneficial to a limited extent, imposes at least some of the limitations and inefficiencies described above in relation to the ditto.com types of image search engines. Applicant""s copending patent application, Ser. No. 09/326,362, now U.S. Pat. No. 6,445,822 does not address how images found in such various data bases, servers, web sites, and the like could be characterized in a manner that requires minimal data or bytes of information for easy and manageable data base storage, quick searching, and ready retrieval for match comparisons.
There is a need for a more automated, high-speed apparatus and method for characterizing images in a manner that can be stored, searched, retrieved, and compared to a reference image with minimal, if any, human intervention or participation.
Accordingly, it is a general object of this invention to provide an improved apparatus and method for characterizing images with information that is unique for each image and that is suitable for encoding, storing, and searching in a data base.
A more specific object of this invention is to provide a method for characterizing shapes in an image quickly and encoding such shape-related characteristics with minimal data so that data storage, searching, and retrieval can be done fast and without extraordinary computer processing power and memory capacity.
Another specific object of the invention is to provide automated search and retrieval of an image from a data base of images based on a reference image.
This and other objects, advantages, and novel features of the invention shall be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following description or may be learned by the practice of the invention. The objects and the advantages may be realized and attained by means of the instrumentalities and in combinations particularly pointed out in the appended claims.
To achieve the foregoing and other objects and in accordance with the purposes of the present invention, as embodied and broadly describe herein, the method of this invention includes characterizing an image for shape content by creating an optical Fourier transform pattern of the image with light energy, spatial filtering the light energy from the Fourier transform pattern with a rotating slit to create a filtered pattern of light energy, detecting intensities of light energy as it is distributed in the filtered pattern at discrete angular orientations of the slit, and storing the intensities of light energy detected in the spatial pattern along with the discrete angular orientation of the slit at which such intensities of light energy are detected. The invention also includes splitting the light energy distributed in the filtered pattern into two beams for detecting by two separate detector arrays that are offset in virtual relation to each other in reference to the filtered pattern in order to avoid losing light energy intensities in spots or zones of light energy that straddle boundaries between two or more individual photosensitive elements in one of the detector arrays. Processing and storing the light energy intensities include combining intensities from individual photosensitive elements with angular orientation of the slit for storage in spaces of a data array that correspond to positions of individual photosensitive elements in the detector array. The intensities can be measured against an intensity threshold and stored in the data base only if they are at least as high as the intensity threshold. A flag, such as one or more distortion levels can be added to the rotation and intensity information stored in the data array.
To further achieve the foregoing objects, the apparatus of this invention includes an optical image shape content characterizer comprising a Fourier transform lens for creating a Fourier transform pattern of light energy from the image at the focal plane of the lens, a spatial filter with a rotating slit positioned in the focal plane of the lens to pass only light energy from the Fourier transform pattern at angular orientations of the slit, a photodetector positioned to detect light passed by the spatial filter, and a spatial light modulator with an associated coherent light source. The spatial light modulator is addressable to produce an image with coherent light from the associated coherent light source and to project the image with the coherent light through the Fourier transform lens. The photodetector preferably includes two detector arrays of individual photosensitive elements, and a beam splitter projects a portion of the filtered pattern to one of the detector arrays and another portion to the other detector array. The detector arrays are virtually offset in relation to each other in reference to the filtered pattern. A comparator circuit (or software performing comparator function) selects intensity from among an individual photosensitive element in one of the arrays and a cluster of virtual partially juxtaposed photosensitive elements in the other array. A RIXel data base array is used to store intensities together with rotational orientation and, optionally, a searchable flag, such as a distortion factor.